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In the context of the ATLAS experiment there is growing evidence of the importance of different 
kinds of Meta-data including all the important details of the detector and data acquisition that are 
vital for the analysis of the acquired data. The Online BookKeeper (OBK) is a component of ATLAS 
online software that stores all information collected while running the experiment, including the 
Meta-data associated with the event acquisition, triggering and storage. The facilities for acquisition 
of control data within the on-line software framework, together with a full functional Web interface, 
make the OBK a powerful tool containing all information needed for event analysis, including an 
electronic log book. 

In this paper we explain how OBK plays a role as one of the main collectors and managers of 
Meta-data produced on-line, and we'll also focus on the Web facilities already available. The usage of 
the web interface as an electronic run logbook is also explained, together with the future extensions. 

We describe the technology used in OBK development and how we arrived at the present level 
explaining the previous experience with various DBMS technologies. The extensive performance 
evaluations that have been performed and the usage in the production environment of the ATLAS 
test beams are also analysed. 



I. INTRODUCTION 

Experiments in High Energy Physics (HEP) are be- 
coming increasingly more complex. The construction 
of a new particle accelerator and associated detectors 
is a technological challenge that also encompasses the 
development of an associated software system. 

The Large Hadron Colider (LHC) currently being 
built at the European Organization for Nuclear Re- 
search (CERN), will support four different detectors 
installed. The ATLAS 1] (A Toroidal LHC Appara- 
tus) at LHC will require a complex trigger system. 
This trigger will have to reduce the original 40MHz 
of p-p interaction rate to a manageable lOOHz for 
storage. The total mass storage, including raw, re- 
constructed, simulated and calibration data exceeds 1 
PetaByte per year. The three major types of data to 



be stored by ATLAS are: 

• raw event data, data collected by the detector 
resulting from the particle collisions (events) 

• conditions and Meta-data, which includes cal- 
ibration constants, run conditions, accelerator 
conditions, trigger settings, detector configura- 
tion, and Detector Control System (DCS) condi- 
tions that determine the conditions under which 
every physics event occurred. These conditions 
are stored with an interval of validity (typically 
time or run number) and retrieved using time 
(or run number) as a key Also the DAQ 
status and time evolution of the Configuration 
Database is included in this type of data. 

• reconstructed data, corresponding to objects 
with physical meaning (e.g. electrons, tracks. 
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etc.) that are the result of applying software 
algorithms to the raw data, taking into consid- 
eration the conditions under which the raw data 
was taken. 

An important part of the conditions database data 
is associated with the Trigger and Data Aquisition 
System 3] (TDAQ). Through the TDAQ system flow 
very different types of data (e.g. calibration and align- 
ment information, conflguration databases informa- 
tion) U that appear as conditions data for storage. 
Some types of this data, such as the accelerator's 
beam parameters, detector configuration, test-beam 
table position, are recorded by a specific component 
of the TDAQ system, the Onhne Bookkeeper (OBK). 
The OBK is important as a source of data for the con- 
ditions database, as an entry point to analysis jobs on 
the raw data and as a debug resource for the DAQ 
system. For this reason, OBK can be qualified as one 
of the biggest collectors and managers of Meta-data 
produced online for the ATLAS experiment. The aim 
of the OBK is to store information describing the data 
acquired by the DAQ and to provide offline access to 
this information Q . It is also a powerful tool for users 
in the control room who can use it as a Run log book 
to attach their comments, or other types of support 
information. 



II. OBK AND THE ONLINE SOFTWARE 

This section will provide an overview of how OBK 
works in the Online Softare framework. As OBK is 
a software package of the Online Software system for 
the ATLAS TDAQ, the architecture of both. Online 
Software and OBK, will be briefly described. 



A. The Online Software architecture 

The role of the Online Software is to provide 
to other TDAQ systems, configuration, control and 
monotoring services. It does not include the process- 
ing and transportation of physics data. All packages 
of the Online Software create a framework generic 
enough to allow supervision of many distinct data 
taking configurations. From the architectural point 
of view there are three different group of components: 
group of components are the Configuration, Con- 
trol and Monitoring. Table shows these main 
packages and their components. 



TABLE I: Online software packages and its components 



Configuration 



Configuration Databases 
Online Bookkeeper 



Control 



Run Control 
DAQ Supervisor 
Process Manager 
Ressource Manager 
Integrated Graphical User Interface 



Monitoring 



Information Service 
Message Reporting System 
Online Histograming 
Event Monitoring 



diagram of OBK in which are the other Online pack- 
ages that OBK interacts with. Namely, the Informa- 
tion Service (IS), Message Reporting System (MRS) 
and Configuration Databases (Conf. DB). 

Both IS and MRS use the IPC (Inter Process Comu- 
nication) packagejB, CORBA implementation, as mes- 
saging backbone |3| • MRS provides the facility which 
allows all software components in the ATLAS DAQ 
system and related processes to report error messages 
to other components of the distributed TDAQ sys- 
tem. IS provides the possibility for inter-application 
information exchange in the distributed environment. 



Online Software 





Conf. DB 




MRS 



Online tools 




B. The OBK architecture 



FIG. 1: Generic OBK architecture. 



The OBK is part of the Configuration group. A 
first prototype was developed 4 years ago and since it 
has evolved significantly. Figure ^ shows a simplified 



These three components of the Online SW are the 
providers of information to OBK databases. The data 
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stored in OBK databases the information will be au- 
tomatically available worldwide. This process begins 
with the data acquisition process in the distributed en- 
vironment and ends up with the final users that will 
query the database for the most relevant informations 
about each data taking period. 

Figure also shows schematically a general archi- 
tecture of OBK. The OBK acquisition software sub- 
scribes to relevant MRS and IS servers in order to 
receive MRS /IS massages exchanged between compo- 
nents via CORBA/IIOP callbacks. The information 
is then stored using an per-run basis philosophy. The 
information stored includes the date/time of each run, 
the basic physics parameters, the status of a run - if 
it was a successfully completed or not, etc. 

The data is then made available for offline usage. 
The users dispose of a set of tools like the Web inter- 
face and the C-I--I- API which can be used to interact 
with OBK databases. They can retrieve data for a 
particular Run, or a set of Runs, or append new rel- 
evant information if needed. The web browser can 
display all the information of a Run including all IS 
informations stored, and with the C++ API there is 
also the facility to search for instances of a particu- 
lar IS class parameter or to cicle all the run headers 
(StartDate, EndDate, TriggerType, etc.) in a parti- 
tion. 



III. PROTOTYPING WITH OBK 

During the last 4 years, three different OBK pro- 
totypes were implemented. The aim was both to 
learn with the experience while trying to implement 
a package that meets the requirements , and to use 
different Data Base Management Systems (DBMS). 
This provided a better understanding of which DBMS 
most fulfills the OBK needs inside the Online Soft- 
ware framework. With this multi-technology ap- 
proach we're gaining technological expertise about dif- 
ferent DBMS technologies and are then able to make 
a solid recommendation for a technological and design 
solution for a production bookkeeper tool. This pro- 
totyping approach seems to achieve very good results 
in a long term experience like this one because each 
of the new OBK prototypes provides more function- 
ality and performance gains over its predecessor. All 
of the prototypes use C++ as programming language 
and the same basic architecture but different DBMS 
for the persistent storage. 

A. Objectivity/DB based 

At the begining, LHC experiments selected Objec- 
tivity/DB for the official DBMS persistent storage. 
An OBK implementation using this DBMS was then 
the first natural choice. 



This prototype was designed to take full advan- 
tage of the pure Object Oriented (00) model of the 
DBMS. The model used with the Objecivity/DB is or- 
ganized in federations, databases, containers and ob- 
ject, OBK was able to map the data coming from the 
TDAQ system in a repository structured in such a 
way. 

The OBK Objectivity/DB based prototype was 
used in 2000 testbeams with success. For data re- 
trieval a web browser was also developed. This allows 
users to get in 'touch' with the data in a very natural 
fashion. Since 2001, this prototype was abandoned 
and is no longer maintained. 



B. OKS based 

The second prototype implemented uses OKS as a 
persistency mechanism. OKS is a C-l — h based, in- 
memory persistent object manager developed as a 
package inside the Online Software OKS is also 
00 but not so sophisticated as Objectivity/DB. The 
data files used for storage are created as XML files. 
They are stored in the file system which can be lo- 
cal, AFS or NFS. The access to the data is done by 
reading directly the files and not through a centralized 
server. The usage of XML files to store data is a very 
interesting feature because they are human readable 
and also highly portable. 

The main reasons that led to this implementa- 
tion were the fact that OKS is Open Source software 
(which makes it usable in any place without having 
any problems with licencing) and also that this DBMS 
is lighter and more oriented to systems with very high 
demands in terms of performance than the Objectiv- 
ity/DB. There are of course some disadvantages of us- 
ing OKS, mainly the lack of features that other DBMS 
provide like transactions. 

The persistent object schema of this prototype is 
very similar to the first one beacause it is also 00 fea- 
tured and the intrinsic data storage philosophy can be 
very similar. A web browser with the same approach 
as the one from Objectivity is also provided. 

This prototype provides extended features while 
compared to the first one, such as more programs 
to control the databases and a full featured C++ 
API. All it's new improvements were used in 2001 test 
beams at CERN by the users. It behaved well and ac- 
quired several Megabytes of data to the local file sys- 
tem of a machine and later on AFS. Despite the good 
behavior from OBK the problem of data dispersion 
and consistency soon arrived because this prototype 
uses several XML files to store information about each 
Run. 
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C. MySQL based 

This is the only OBK prototype that uses a Rela- 
tional DBMS. MySQL 9] is a well known, fast and 
reliable Open Source DBMS. It started a new phase 
in the OBK development - The phase of the relational 
model. The decision to implement a package such as 
OBK using MySQL was driven from the power of its 
underlying SQL engine and also due to the desire of 
trying a relational approach to OBK databases. Tech- 
nically a new database schema was implemented to al- 
low mapping of data coming mainly from 00 sources 
forcing us to completely redesign the internal struc- 
ture of OBK. We have achieved a mapping between 
an 00 and a Relational schema that is suitable for 
OBK needs and started to use it. 

This prototype provides all the features of the pre- 
vious ones, plus further enhancements regarding the 
users needs and also the very important aspect of per- 
formance. In this implementation the concept of log 
book was also introduced and successfully deployed 
(see section HV BI for details) and successfully used. 

The MySQL implementation was used successfully 
in the 2002 test beam. It recorded more than 1 GByte 
of data, including data coming from the DCS. 



IV. OBK INTERFACES FOR DATA 
RETRIEVAL 

OBK provides a set of interfaces that allow users to 
interact with the databases. There are also available 
some tools coded in C-I--I- which make some tasks very 
easy to execute. In this section the focus will be on 
the C-I--I- interfaces (the Query API) and on the Web 
interface used to store information directly related to 
the users. 



A. C+-I- Query API 

Both OKS and MySQL prototype are distributed 
with a C-I--I- Query API. This API exposes meth- 
ods that allow data retrieval in a very user oriented 
approach. The API encapsulates all the necessary 
mechanisms to get the correct values from the OBK 
databases. The users do not need any special skill (like 
how to perform SQL queries) regarding the database 
schema. This also allows to preserve the integrity of 
the database because it doesn't allow users to ma- 
nipulate it directly in what concerns for example the 
addiction of new information retated to a Run. 



B. Web Browser 

The latest version of the OBK browser includes 
several functionalities making it a powerful tool. It 
provides not only the functionalities of displaying 
the data coming from the OBK data acquisition 
process just after it was collected through the Onhne 
System but also the possibility of behaving like a Run 
Log Book. It was built with an excellent searching 
mechanism oriented to the final users which will be 
the Physicists. 
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2: OBK browser - search mechanism. 



Figure El shows all the options of this search 
mechanism. There are a set of options on which the 
users can base their criteria of selection such as: 

• RunStatus Good/Bad 

• MELxEnvents The maximum number of events 
of the Runs 

• Start Date/End Date. For the Start and end 
of a Run. This allows to 'map' the obk run bases 
data type in a time interval 

• BeamType Muons, Electrons, etc. 

• TriggerType Cosmic, Calibration, Physics 

Sorting options are also included. The OBK data dis- 
play in this browser was driven by the need of a clear 
and very user friendly interface where the data would 
be easily accessible to the users. After the selection of 
all the criteria that the user wants to meet, the result 
will appear in another web page similar to the one in 
Figure 13 

The result presents to the users some relevant in- 
formation about each Run: Partition to which each 
Run belongs; Run Number; Start and End Date of 
the Run; Run Status; Number of Events; Maximum 
Number of Events; Trigger Type, Detector Mask and 
Beam Type. It is also possible to display more specific 
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FIG. 3: OBK browser - result of the search. 



information, like for example which messages from the 
MRS were transferee! between the various components 
of the Online Software. This is accessible by following 
the link provided for each Run. 





FIG. 5: OBK browser - adding a comment. 



The OBK browser itself also provides administra- 
tive tools, for example to create databases with the 
correct structure for OBK, and gives a set of op- 
tions for user management: authentication, permis- 
sions, etc. 



V. TESTING 




FIG. 4: OBK browser - more information about a partic- 
ular Run. 

Figure^ shows a typical page generated when se- 
lecting the option to display more detailed informa- 
tion on a particular Run. Through this new page it is 
possible to browse information including the messages 
from the MRS, from the IS and the users comments 
and attached files. There are two different approaches 
to storing comments in OBK databases: 

• using binary programs provided for both online 
and offline comments 

• Using the web browser 

Adding comments through the web browser gives 
also the option of attaching any type of files to a com- 
ment. This allows users to add information that they 
think might be relevant to the Run. Afterwards every 
one can see the comments and their respectively at- 
tached files. Some types of files are supported and will 
immediately be displayed in a different window. File 
types that are not supported will have to be down- 
loaded and the appropriate program must be used to 
open them. 



For the evaluation of the various prototypes the fo- 
cus was to analyse the different functionalites, how 
easy it was to map the data coming from the Online 
Sytem or on how complex the code of each one be- 
came. On all of these issues a set of scalablity and 
performance tests were addressed. One other objec- 
tive of these tests was also to evaluate if OBK can 
handle all the information produced in the final sys- 
tem. 

In Figure|H|it can be seen the time to store a typical 
IS message in function of the number of OBK data ac- 
quisition programs running simultaneously. This test 
was one of the tests performed in the scalability con- 
text regarding the final system. 

A comparison between the three prototypes was 
also addressed. This test was performed with a typi- 
cal Start of Run message coming from the MRS. The 
time for this test is not only the time spent to store 
the message itself but also all operations that this im- 
plies. This operation includes the creation of a new 
Run in the database with all associated operations 
like the creation of new files (in the OKS prorotype) 
or containers (in the case of the Objectivity/DB pro- 
totype). More details about this tests can be found in 
The MySQL prototype proved to be the faster 
one while the Objectivity/DB is the slowest. It was 
clear from the test that there is a dependency on the 
Run number for the time spent to store a message of 
this kind in case of both Objectivity/DB and OKS 
prototypes. The slope of the OKS line is less than 
the Objectivity/DB but when a transaction becomes 
commited, the Objectivity/DB prototype gets better. 
We attribute these results not only to the evolution in 
the design from prototype to prototype but also be- 
cause MySQL provides a faster engine that makes the 
time to store these messages negligible when compared 



TUGP007 



6 



CHEP03 March 24-28, 2003 La Jolla, California 



Time to store an IS message vs N. OBKs 
MifSQL implementation 
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FIG. 6; OBK tests - scalability test performed to the 
MySQL implementation. 



with the other prototypes. Some other performance 
results are displayed in tables HTl and IIIII 



TABLE II: Time (in miliseconds) to Start of Run, End of 
Run, Comment and a typical IS message for OBK/OKS 
prototype. 



Platform 


Start Run 


End Run 


Comment 


IS 


Linux/egcsl.l 


77-259 


48-245 


0,3 


2,7 


Linux/gcc2.96 


47-196 


29-184 


0,3 


1,9 



The results presented for the Start of Run, End of 
Run and Comment, represent the the minimum value 
and the maximum observed during the tests. Tests 
were performed to a maximum of 500 Runs. For the 
IS time it's the mean time to store a IS message from 
OBK because it was observed that there was no sig- 
nificant growth. 



TABLE III: Time (in miliseconds) to Start of Run, 
End of Run, Comment and a typical IS message for 
OBK/MySQL prototype. 



Platform 


Start Run 


End Run 


Comment 


IS 


Linux/egcsl.l 


0,004 


0,010 


0,002 


0,011 


Linux/gcc2.96 


0,018 


0,021 


0,004 


0,020 



More information about these tests can be found 
in OBK test report This document includes a 

detailed description of the test procedure and other 
results such as functionality tests. 



VI. SUMARY AND FUTURE WORK 

In this paper we presented a general overview of 
our experience of using different DBMSs in prototyp- 
ing OBK in the ATLAS Online Software framework. 
Since the beginning of the project we tried to under- 
stand the problem of bookkeeping for the ATLAS ex- 
periment. For that reason, OBK evolved and it now 
provides some tools that can be qualified as Run log 
book tools. The last version of OBK which is the most 
pcrformant and robust one is the result of this expe- 
rience. But OBK work is still in progress. Included 
in the list of future improvements for OBK are: 

• to make it DBMS independent in order to have 
a more general and dynamic architecture; 

• create a set of new tools to extend the actual 
existing ones for the log book approach; 
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